Renovate Yourself: Calibrating Feature Representation of Misclassified Pixels for Semantic Segmentation

Existing image semantic segmentation methods favor learning consistent representations by extracting long-range contextual features with the attention, multi-scale, or graph aggregation strategies. These usually treat misclassified and correctly classified pixels equally, hence misleading optimization process causing inconsistent intra-class pixel feature in embedding space during learning. In this paper, we propose auxiliary representation calibration head (RCH), which consists of decoupling, prototype clustering, error modules a metric loss function, to calibrate these error-prone for better consistency performance. RCH could be incorporated into hidden layers, trained together networks, decoupled inference stage without additional parameters. Experimental results show that our method significantly boost performance current on multiple datasets (e.g., outperform original HRNet OCRNet 1.1% 0.9% mIoU Cityscapes test set). Codes are available at https://github.com/VipaiLab/RCH.